Eliciting Mental Models

Title: Using the repertory grid and laddering technique to determine the user's evaluative model of search engines. Purpose This study explores a method for the determination of users' representations of search engines, formed during their interaction with these systems. The purpose is to determine the extent to which these elicited 'mental models' indicate the system aspects of importance to the user and from this their evaluative view of these tools. The repertory grid technique is used to elicit a set of constructs that define facets within the mental model of an individual. A related technique of laddering then considers each of the user's constructs to determine the reasons for its importance within the user's mental model. Findings The model derived from the qualitative data comprises three hierarchical strata and conveys the interrelations between basic system description, evaluative description, and the key evaluations of ease, efficiency, effort and effectiveness. Two additional layers relating to the perceived process and the experience of emotion are also discussed. Ten participants is considered to be optimum for obtaining constructs in a repertory grid, but limits the findings to the context of the user group and the systems used in this study. Originality/value of the paper The methodology has not previously been used to determine mental models of search engines and from these to understand users' evaluative view of systems. The resulting model of key evaluations with the conjunctions of procedural elements suggests a framework for further research to evaluate search engines from the user perspective. Introduction Fundamental to the progression of Web retrieval system development is the need to understand the way the tools are perceived by the end-users. Unlike the target audience of systems such as DIALOG, web searchers form a large body of 'ordinary' users, with little or no formal information retrieval (IR) training. Although studies such as Spink et al (2001) indicate that users' interactions with search engines are often brief, users will nevertheless form an opinion of the tools they have used, and the opinions will inform any subsequent choice of search engine. These impressions are unlikely to be formed in the abstract and may be underpinned by the users' 'mental models', the term often used for the user's internal representation of the system resulting from their interaction. This paper explores the value of these mental models in determining the criteria on which to base the evaluation of interactive retrieval systems …